What is a sensible payload schema design for a document search system that needs to support filtering by date range, category, and access permissions?

The schema should use the payload type that matches the filter's access pattern, and each filterable field should have a payload index with a matching schema. For date range filtering, use the datetime payload type and a datetime index - this gives efficient range queries and lets the planner prune segments whose date ranges do not overlap. For category filtering, use a keyword field and a keyword index - keyword indexes are hash-based and support exact-match must and must_not filters. For access permissions, which are typically a list of roles or groups, use a keyword-array field and a keyword index; Qdrant's keyword index supports array values and can match if any element matches (should) or require all (must). The schema should also include a stable point ID scheme so that a document can be reconstructed from a payload lookup if needed, and a reference to the source document so that chunks or multiple embeddings of the same document can be traced back.

The mechanism behind each choice is the index structure. A datetime index stores timestamps in a sorted structure that supports range queries and min/max statistics, which is what enables both the range filter and the segment-level pruning. A keyword index maps each distinct value to a posting list of point IDs, so an exact-match filter resolves to a set lookup; for an array field, the same structure supports membership tests. A field that is filtered with a range but indexed as a keyword will not get range pruning and will be slower. A field that is filtered with exact match but not indexed will require a scan of the payload for every candidate. The schema must therefore reflect how the field is actually queried, not just what type the value happens to be. Access permissions are a good example: if they are stored as a list of roles and the query asks 'which documents can this user see', the filter is a should over the roles, which a keyword index on an array field handles efficiently. If the query asks 'which documents require all of these roles', the filter is a must over the roles, which the same index handles. If permissions are hierarchical or group-based, the application must expand the user's group memberships into a list before querying.

The trade-off is between index coverage and memory/write cost. Every payload index consumes memory and slows down upserts because each write must update the index. Indexing every field defensively is expensive; indexing too few fields makes filters slow. The right approach is to base the indexing decision on actual query patterns: index the fields that appear in production filters, and leave the rest unindexed. The common mistake is to store a date as a string instead of a datetime, which prevents range queries from using the datetime index and forces a slower scan. The second mistake is to store categories as a single comma-separated string instead of an array, which prevents exact-match filtering per category. The third mistake is to store permissions as a nested structure that the filter cannot address efficiently. The fourth mistake is to forget the tenant discriminator and then add it later, which requires reindexing. Version note: the datetime type, the keyword-array support, and the is_tenant flag have evolved across Qdrant releases. Verify the supported payload types and index schemas on your version before designing the schema.

Version-dependent: the supported payload schema types and the keyword-array behavior have evolved across Qdrant releases. The datetime format (ISO 8601) is stable, but the exact range API and the is_tenant flag are recent additions. Verify the schema types and the index options on your version before designing the schema, and test the filter behavior with a small fixture to confirm that the expected filtering works.